Control System And Method For Robotic Motion Planning And Control

ABSTRACT

A robotic system includes a robotic vehicle having a propulsion system, one or more sensors that image data representative of an external environment, and a controller that determines a waypoint for the robotic vehicle to move toward. The controller determines limitations on movement of the robotic vehicle toward a waypoint. The limitations are based on the image data. The controller controls the propulsion system to move the robotic vehicle to the waypoint subject to the limitations on the movement to avoid colliding with one or more objects. The controller determines one or more additional waypoints subsequent to the robotic vehicle reaching the waypoint, determines one or more additional limitations on the movement of the robotic vehicle toward each of the respective additional waypoints, and control the propulsion system of the robotic vehicle to sequentially move the robotic vehicle to the one or more additional waypoints.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application claims priority to U.S. Provisional Application No.62/342,448, filed 27 May 2016, the entire disclosure of which isincorporated herein by reference.

FIELD

The subject matter described herein relates to systems and methods forautonomously controlling movement of a device.

BACKGROUND

The challenges in the modern vehicle yards are vast and diverse.Classification yards, or hump yards, play an important role asconsolidation nodes in vehicle freight networks. At classificationyards, inbound vehicle systems (e.g., trains) are disassembled and thecargo-carrying vehicles (e.g., railcars) are sorted by next commondestination (or block). The efficiency of the yards in part drives theefficiency of the entire transportation network.

The hump yard is generally divided into three main areas: the receivingyard, where inbound vehicle systems arrive and are prepared for sorting;the class yard, where cargo-carrying vehicles in the vehicle systems aresorted into blocks; and the departure yard, where blocks of vehicles areassembled into outbound vehicle systems, inspected, and then depart.

Current solutions for field service operations are labor-intensive,dangerous, and limited by the operational capabilities of humans beingable to make critical decisions in the presence of incomplete orincorrect information. Furthermore, efficient system level-operationsrequire integrated system wide solutions, more than just point solutionsto key challenges. The nature of these missions dictates that the tasksand environments cannot always be fully anticipated or specified at thedesign time, yet an autonomous solution may need the essentialcapabilities and tools to carry out the mission even if it encounterssituations that were not expected.

Solutions for typical vehicle yard problems, such as brake bleeding,brake line lacing, coupling cars, etc., can require combining mobility,perception, and manipulation toward a tightly integrated autonomoussolution. When placing robots in an outdoor environment, technicalchallenges largely increase, but field robotic application benefits bothtechnically and economically.

One challenge in using automated robotic systems to perform maintenanceof the vehicles in the yard is ensuring that the robotic systems safelymove through the yard. For example, safeguards are needed to ensure thatthe robotic systems do not collide with other objects (stationary ormoving) and that the robotic systems are able to respond to adynamically changing environment (e.g., where an object moves into thepath of a moving robotic system), while also attempting to ensure thatthe robotic systems move toward locations for performing the vehiclemaintenance along efficient paths (e.g., the shortest possible path orthe path that is shorter than one or more other paths, but not allpaths).

BRIEF DESCRIPTION

In one embodiment, a robotic system includes a robotic vehicle having apropulsion system configured to propel the robotic vehicle, one or moresensors configured to be disposed onboard the robotic vehicle and toobtain image data representative of an external environment, and acontroller configured to be disposed onboard the robotic vehicle and todetermine a waypoint for the robotic vehicle to move toward. Thewaypoint is located between a current location of the robotic vehicleand a final destination of the robotic vehicle. The controller also isconfigured to determine limitations on movement of the robotic vehicletoward the waypoint. The limitations are based on the image data. Thecontroller is configured to control the propulsion system to move therobotic vehicle to the waypoint subject to the limitations on themovement to avoid colliding with one or more objects. The controlleralso is configured to determine one or more additional waypointssubsequent to the robotic vehicle reaching the waypoint, determine oneor more additional limitations on the movement of the robotic vehicletoward each of the respective additional waypoints, and control thepropulsion system of the robotic vehicle to sequentially move therobotic vehicle to the one or more additional waypoints.

In one embodiment, a method includes obtaining image data representativeof an environment external to a robotic system, determining a waypointfor the robotic system to move toward, the waypoint located between acurrent location of the robotic system and a final destination of therobotic system, determining limitations on movement of the roboticsystem toward the waypoint. The limitations are based on the image data,controlling a propulsion system of the robotic system to move therobotic system to the waypoint subject to the limitations on themovement to avoid colliding with one or more objects, determining one ormore additional waypoints subsequent to the robotic system reaching thewaypoint, determining one or more additional limitations on the movementof the robotic system toward each of the respective additionalwaypoints, and controlling the propulsion system of the robotic systemto sequentially move the robotic system to the one or more additionalwaypoints.

In one embodiment, a robotic system includes a robotic vehicle having apropulsion system configured to propel the robotic vehicle, one or morestructured light sensors configured to be disposed onboard the roboticvehicle and to obtain point cloud data representative of an externalenvironment, and a controller configured to be disposed onboard therobotic vehicle and to determine a waypoint for the robotic vehicle tomove toward. The waypoint is located between a current location of therobotic vehicle and a brake lever of a vehicle. The controller also isconfigured to determine limitations on movement of the robotic vehicletoward the waypoint. The limitations are based on the point cloud data.The controller is configured to control the propulsion system to movethe robotic vehicle to the waypoint subject to the limitations on themovement to avoid colliding with one or more objects.

BRIEF DESCRIPTION OF THE DRAWINGS

The present inventive subject matter will be better understood fromreading the following description of non-limiting embodiments, withreference to the attached drawings, wherein below:

FIG. 1 illustrates one embodiment of a robotic system;

FIG. 2 illustrates a flowchart of a method or state diagram of operationof a controller of the robotic system shown in FIG. 1 in directingmovement of the robotic system according to one embodiment;

FIG. 3 illustrates one example of sensor data that can be examined bythe controller shown in FIG. 1 to determine how to autonomously move therobotic system also shown in FIG. 1; and

FIG. 4 illustrates one example of a waypoint location that can bedetermined for the robotic system shown in FIG. 1.

DETAILED DESCRIPTION

FIG. 1 illustrates one embodiment of a robotic system 100. The roboticsystem 100 may be used to autonomously move toward, grasp, and actuate(e.g., move) a brake lever or rod on a vehicle in order to change astate of a brake system of the vehicle. For example, the robotic system100 may autonomously move toward, grasp, and move a brake rod of an airbrake system on a rail car in order to bleed air out of the brakesystem. The robotic system 100 includes a robotic vehicle 102 having apropulsion system 104 that operates to move the robotic system 100. Thepropulsion system 104 may include one or more motors, power sources(e.g., batteries, alternators, generators, etc.), or the like, formoving the robotic system 100. A controller 106 of the robotic system100 includes hardware circuitry that includes and/or is connected withone or more processors (e.g., microprocessors, field programmable gatearrays, and/or integrated circuits) that direct operations of therobotic system 100.

The robotic system 100 also includes several sensors 108, 109, 110, 111,112 that measure or detect various conditions used by the robotic system100 to move toward, grasp, and actuate brake levers. The sensors 108-111are optical sensors, such as cameras, infrared projectors and/ordetectors. While four optical sensors 108, 110 are shown, alternatively,the robotic system 100 may have a single optical sensor, less than fouroptical sensors, or more than four optical sensors. In one embodiment,the sensors 109, 111 are RGB cameras and the sensors 110, 112 arestructured-light three-dimensional (3-D) cameras, but alternatively maybe another type of camera.

The sensor 112 is a touch sensor that detects when a manipulator arm 114of the robotic system 100 contacts or otherwise engages a surface orobject. The touch sensor 112 may be one or more of a variety oftouch-sensitive devices, such as a switch (e.g., that is closed upontouch or contact), a capacitive element (e.g., that is charged ordischarged upon touch or contact), or the like. Alternatively, one ormore of the sensors 108-112 may be another type of sensor, such as aradar sensor, LIDAR sensor, etc.

The manipulator arm 114 is an elongated body of the robotic system 100that can move in a variety of directions, grasp, and pull and/or push abrake rod. The controller 106 may be operably connected with thepropulsion system 104 and the manipulator arm 114 to control movement ofthe robotic system 100 and/or the arm 114, such as by one or more wiredand/or wireless connections. The controller 106 may be operablyconnected with the sensors 108-112 to receive data obtained, detected,or measured by the sensors 108-112.

The robotic system 100 can include a communication device 116 thatcommunicates with an off-board control unit 118. The communicationdevice 116 can represent one or more antennas and associatedtransceiving circuitry, such as one or more modems, transceivers,receivers, transmitters, etc. The control unit 118 can representhardware circuitry that includes and/or is connected with one or moreprocessors (e.g., microprocessors, field programmable gate arrays, orintegrated circuits) that receives user input to remotely controlmovement and other operation of the robotic system 100. In oneembodiment, the control unit 118 also represents one or more inputdevices, such as joysticks, touchscreens, styluses, keyboards, etc., toallow a user to remotely control movement and other operations of therobotic system 100. The control unit 118 also can include one or moreantennas and associated transceiving circuitry to allow wirelesscommunication with the communication device 116 of the robotic system100. Alternatively or additionally, the communication device 116 of therobotic system 100 may be connected with the control unit 118 by one ormore wired connections to allow for remote control of the robotic system100 via the wired connection(s).

FIG. 2 illustrates a state diagram 200 of operation of the controller106 in directing movement of the robotic system 100 shown in FIG. 1according to one embodiment. The state diagram 200 can represent aflowchart of a method for controlling movement of the robotic system100, and may represent or be used to create software that directsoperation of the controller 106.

The controller 106 may operate using the method represented by the statediagram 200 to move the robotic system 100 between or among differentlocations (e.g., in vehicle yards or other locations) to perform tasks,such as maintenance, inspection, repair, etc., of the vehicles. Thecontroller 106 may operate in different operational modes. One mode canbe referred to as an autonomous navigation mode and another mode can bereferred to as tele-operation mode. Operations performed or controlledby the controller 106 can be referred to herein as modules. The modulescan represent different sets of functions performed by the same ordifferent processors of the controller 106, and/or can representdifferent hardware components (e.g., processors and associatedcircuitry) performing the functions associated with the respectivemodules.

At 202, the robotic system 100 is in a ready state. The ready state mayinvolve the robotic system 100 being stationary and prepared to beginmovement. The controller 106 may monitor the communication device 116(or wait for a signal from the communication device 116) to indicatewhether the robotic system 100 is to begin movement. The controller 106may receive an input signal from the control unit 118 via thecommunication device 116 and/or from an input device of the roboticsystem 100 (e.g., one or more buttons, knobs, switches, touchscreens,keyboards, etc.). Responsive to receiving the input signal, thecontroller 106 may determine whether the input signal indicates that therobotic system 100 is to operate in the autonomous navigation mode (alsoreferred to as “Autonomous NAV” in FIG. 2; e.g., the operations orstates shown in connection with 201 in FIG. 2) or the tele-operation (ormanual navigation or remote control) mode (also referred to as “ManualNAV” in FIG. 2; e.g., the operations or states shown in connection with203 in FIG. 2). The input signal may indicate that the robotic system100 is to operate in the tele-operation mode if the input signalindicates movement of the input device of the control unit 118, such asmovement of a joystick or other input. The input signal may indicatethat the robotic system 100 is to operate in the autonomous navigationmode if the input signal indicates other actuation of the input deviceof the control unit 118, such as selection of an input that indicatesautonomous operation.

If the controller 106 determines that the robotic system 100 is tooperate in the tele-operational mode 203, then flow of the method orstate diagram 200 may proceed toward 204. If the controller 106determines that the robotic system 100 is to operate in the autonomousnavigation mode 201, then flow of the method or state diagram 200 mayproceed toward 208.

At 204, the robotic system 100 determines if a permissive signal to movehas been generated or provided. The permissive signal may be generatedor provided by a deliberation module of the controller 106. Thedeliberation module receives input from the control unit 118, such asmovement of a joystick or other input that indicates a direction ofmovement, speed, and/or acceleration of the robotic system 100. Thedeliberation module of the controller 106 also examines data or otherinformation provided by one or more of the sensors 108-112 to determinewhether movement, as requested or dictated by the input received fromthe control unit 118, is feasible and/or safe. For example, thedeliberation module of the controller 106 can obtain two dimensional(2D) image data (e.g., 2D images or video) from the sensors 109 and/or111, three dimensional (3D) image data (e.g., 3D images or video, pointclouds, etc.) from the sensors 108 and/or 110, and/or detection ofengagement or touch of an object from the sensor 112. The deliberationmodule can examine this data to determine if the movement requested bythe control unit 118 can be performed without the robotic system 100colliding with another object or operating in another unsafe manner. Forexample, the deliberation module can examine the 2D and/or 3D image datato determine if one or more obstacles remain in the movement pathrequested by the input. As described in more detail below, the imagedata provided by one or more of the sensors 108-111 can be used todetermine whether any objects are in the path of the robotic system 100.

If the controller 106 (e.g., the deliberation module) determines thatthe robotic system 100 can move according to the input provided by thecontrol unit 118 at 202, then flow of the method or state diagram 200continues toward 206. Otherwise, the method or state diagram 200 mayremain at 204 until permission to move is received from or otherwiseprovided by the deliberation module of the controller 106.

At 206, the robotic system 100 moves according to the input provided byor otherwise received from the control unit 118. For example, responsiveto receiving permission to move the robotic system 100 according to theinput provided by the control unit 118, the controller 106 may generatecontrol signals that are communicated to the propulsion system 104 ofthe robotic system 100 to move the robotic system 100 according to theinput. Upon completion of the movement, the propulsion system 104 maystop moving the robotic system 100 and flow of the method or statediagram 200 may return toward 204.

If, at 204, it is determined that the robotic system 100 is to operatein the autonomous navigation mode 201, then flow of the method or statediagram 200 may proceed toward 208. For example, if the input receivedby the controller 106 from the control unit 118 indicates that therobotic system 100 is to autonomously move, then flow may proceed toward208.

At 208, a navigation module of the controller 106 informs thedeliberation module that autonomous movement of the robotic system 100has been initiated. This can involve the controller 106 from the manualnavigation mode to the autonomous navigation mode. The robotic system100 may remain stationary and optionally prohibit movement of therobotic system 100 until confirmation of the change from the manual toautonomous navigation mode has been received. This confirmation may beprovided from the deliberation module of the controller 106.

Responsive to receiving confirmation that the autonomous movement of therobotic system 100 has been initiated, at 210, a determination is madeas to whether the robotic system 100 can move. This determination mayinvolve examining data provided by one or more of the sensors 108-112,in addition to or exclusive of other data provided to or accessible bythe controller 106. For example, in addition to the image data providedby one or more of the sensors 108-111, the controller 106 may access amemory or database (not shown) onboard or off-board the robotic system100 (e.g., via the communication device 116). The controller 106 canobtain information such as a current location of the robotic system 100(e.g., via a global positioning system receiver or data), locations ofvehicles in the vehicle yard, numbers of vehicles in a vehicle consistthat the robotic system 100 is to move alongside, known or designatedlocations of objects in or around the robotic system 100, etc.

A perception module of the controller 106 can examine the sensor dataand/or other data to determine how to autonomously move the roboticsystem 100. The perception module can examine this data to determine howto safely and efficiently move the robotic system 100 withoutintervention (or at least additional intervention) from a humanoperator.

FIG. 3 illustrates one example of sensor data 300 that can be examinedby the controller 106 to determine how to autonomously move the roboticsystem 100. The sensor data 300 is a point cloud that representslocations of different points in 3D space. The sensor data 300 may beobtained from a structured light sensor, such as a Microsoft KINECTcamera device or other structured light sensor. The point cloudindicates where different objects are located relative to the sensor108, 110 that provided the data used to create the point cloud. Theperception module of the controller 106 can examine the point cloud todetermine if there are any objects that the robotic system 100 couldcollide with. Based on the locations of the points in the point cloud,the perception module of the controller 106 can determine how far theobject is from the sensor that provided the data used to generate thepoint cloud. Optionally, the controller 106 may examine other data, suchas 2D or 3D images obtained by the sensors 108-111, detection of touchas determined by the sensor 112, radar data provided by one or moresensors, or other data, to determine the presence, distance to, relativelocation, etc., of other object(s) around the robotic system 100.

Returning to the description of the method or state diagram 200 shown inFIG. 2, at 210, the perception module examines the data to determinewhether the robotic system 100 can move without colliding with anotherobject (stationary or moving) and, if the robotic system 100 can movewithout a collision, where the robotic system 100 can move. For example,the controller 106 can examine the data to determine allowable limits onwhere the robotic system 100 can move. These limits can includerestrictions on how far the robotic system 100 can move in one or moredirections, how fast the robotic system 100 can move in one or moredirections, and/or how quickly the robotic system 100 can accelerate ordecelerate in one or more directions. The controller 106 may hold off onmoving the robotic system 100 until the determination is made as towhether the robotic system 100 can move and limitations on how therobotic system 100 can move.

At 212, the robotic system 100 autonomously moves. In one embodiment,the navigation module of the controller 106 determines a waypointlocation for the robotic system 100 to move toward. The waypointlocation may be a geographic location that is between a current locationof the robotic system 100 and a final, destination, or goal locationthat the robotic system 100 is moving toward. For example, if therobotic system 100 is to move five meters to a brake lever of a vehiclein order to grasp and pull the brake lever (e.g., to bleed an air brakeof the vehicle), the navigation module may generate control signals tocause the robotic system 100 to move to a waypoint that is fiftycentimeters (or another distance) toward the brake lever from thecurrent location of the robotic system 100, but that is not at thelocation of the brake lever.

FIG. 4 illustrates one example of a waypoint location that can bedetermined for the robotic system 100. The navigation module can use thesensor data (and/or other data described herein) and determine locationsof other objects (“Plane of Railcar” in FIG. 4), the surface on whichthe robotic system 100 is moving (“Ground Plane” in FIG. 4), and/or thewaypoint location to which the robotic system 100 is moving (“Waypoint”in FIG. 4). For example, the point cloud obtained from one or more ofthe sensors 108, 110 can be examined to determine locations of the otherobjects and/or surface shown in FIG. 4.

In one embodiment, the controller 106 can determine the locations ofobjects using the data with simultaneous localization and mapping(SLAM). For example, the controller 106 can use real-timeappearance-based mapping (RTAB-Map) to identify the locations ofobjects.

The navigation module of the controller 106 can generate control signalsto dictate how the robotic system 100 moves toward the waypointlocation. These control signals may designate the direction of movement,the distance that the robotic system 100 is to move, the moving speed,and/or acceleration based on the current location of the robotic system100, the waypoint location, and/or limitations determined by theperception module of the controller 106 (described below). Thenavigation module generates control signals that are communicated to thepropulsion system 104 of the robotic system 100. These control signalsdirect the motors and other components of the propulsion system 104 howto operate to move the robotic system 100.

Movement of the robotic system 100 can be monitored to determine whetherthe movement of the robotic system 100 has or will violate one or morepredefined or previously designated limits. In the illustrated example,the robotic system 100 is not allowed to move more than forty inches(e.g., 102 centimeters). Optionally, another distance limitation orother limitation (e.g., a limitation on an upper or lower speed, alimitation on an upper or lower acceleration, a limitation on adirection of movement, etc.) may be used. If the movement of the roboticsystem 100 reaches or violates one or more of these limitations, flow ofthe method or state diagram 200 can proceed toward 214.

In one embodiment, the controller 106 determines the movements of therobotic system 100 to try and achieve different goals. One goal is tomove the robotic system 100 so as to minimize or reduce the distancebetween the robotic system 100 and the desired location, such as thenext waypoint (relative to moving the robotic system 100 along one ormore, or all, other feasible paths to the next waypoint). Another goalis to keep at least a designated safe distance between the roboticsystem 100 and one or more other objects, such as rail tracks on whichthe vehicles are disposed. The controller 106 can determine commands forthe propulsion system 104 that drive the robotic system 100 toward thenext waypoint and fuse these commands with commands that keep therobotic system 100 away from the vehicles (or other objects), by atleast a designated, non-zero distance (e.g., four inches or tencentimeters). These commands are combined by the controller 106 todetermine a velocity command that will control the propulsion system 104of the robotic system 100 to move. The fusion can be a weighted sum ofthe commands:

cmd _(vel) =α*cmd _(goal) +β*cmd _(safety)  (1)

α+β=1  (2)

where cmd_(vel) represents the velocity command, cmd_(goal) andcmd_(safety) are generated using the artificial potential fieldalgorithm, and α and β are parameters that are tuned or set based on thetask-relevant situations.

At 214, movement of the robotic system 100 is stopped. The navigationmodule of the controller 106 can generate and communicate an alarmsignal to the propulsion system 104 that stops movement of the roboticsystem 100. This signal can direct motors to stop rotating wheels of thevehicle 102 of the robotic system 100 and/or direct a brake of thevehicle 102 to stop movement of the robotic system 100. Flow of themethod or state diagram 200 may then return toward 202.

But, if movement of the robotic system 100 does not reach or violate thelimitation(s), then the robotic system 100 may continue autonomouslymoving toward the waypoint location. The following motion will bedetermined through a decision making process (e.g., motion and energyoptimization). As described above, the navigation module may direct thepropulsion system 104 to move the robotic system 100 to move toward, butnot all the way to, a destination or goal location. Instead, thenavigation module can direct the propulsion system 104 to move therobotic system 100 part of the way to the destination or goal location.

The robotic system 100 moves toward the waypoint location subject to thelimitations described above. Upon reaching the waypoint location, flowof the method or state diagram 200 can return toward 210 from 212. Forexample, the perception module can again determine whether the roboticsystem 100 can move based on the sensor data and/or other data, asdescribed above. At least a portion of the method or state diagram 200may repeat one or more times or iterations between perceiving thesurroundings, determining a subsequent waypoint, determining limitationson movement toward the waypoint, and moving to the waypoint. Eventually,the final waypoint may be at or near the final destination of therobotic system 100.

In one embodiment, a robotic system includes a robotic vehicle having apropulsion system configured to propel the robotic vehicle, one or moresensors configured to be disposed onboard the robotic vehicle and toobtain image data representative of an external environment, and acontroller configured to be disposed onboard the robotic vehicle and todetermine a waypoint for the robotic vehicle to move toward. Thewaypoint is located between a current location of the robotic vehicleand a final destination of the robotic vehicle. The controller also isconfigured to determine limitations on movement of the robotic vehicletoward the waypoint. The limitations are based on the image data. Thecontroller is configured to control the propulsion system to move therobotic vehicle to the waypoint subject to the limitations on themovement to avoid colliding with one or more objects. The controlleralso is configured to determine one or more additional waypointssubsequent to the robotic vehicle reaching the waypoint, determine oneor more additional limitations on the movement of the robotic vehicletoward each of the respective additional waypoints, and control thepropulsion system of the robotic vehicle to sequentially move therobotic vehicle to the one or more additional waypoints.

In one example, the one or more sensors are configured to obtain a pointcloud of the external environment using one or more structured lightsensors.

In one example, controller is configured to determine the limitations onthe movement of the robotic vehicle by determining relative locations ofone or more objects in the external environment based on the image dataand restricting movement of the robotic vehicle to avoid colliding withthe one or more objects.

In one example, the controller is configured to determine thelimitations using simultaneous localization and mapping to restrict themovement of the robotic vehicle.

In one example, the controller also is configured to stop movement ofthe robotic vehicle responsive to the robotic vehicle moving fartherthan a designated, non-zero distance toward the waypoint.

In one example, the final destination of the robotic vehicle is a brakelever of a vehicle.

In one example, the controller also is configured to switch betweenmanual control of the movement of the robotic vehicle and autonomousmovement of the robotic vehicle based on input received from a controlunit disposed off-board the robotic vehicle.

In one embodiment, a method includes obtaining image data representativeof an environment external to a robotic system, determining a waypointfor the robotic system to move toward, the waypoint located between acurrent location of the robotic system and a final destination of therobotic system, determining limitations on movement of the roboticsystem toward the waypoint. The limitations are based on the image data,controlling a propulsion system of the robotic system to move therobotic system to the waypoint subject to the limitations on themovement to avoid colliding with one or more objects, determining one ormore additional waypoints subsequent to the robotic system reaching thewaypoint, determining one or more additional limitations on the movementof the robotic system toward each of the respective additionalwaypoints, and controlling the propulsion system of the robotic systemto sequentially move the robotic system to the one or more additionalwaypoints.

In one example, obtaining the image data includes obtaining a pointcloud using one or more structured light sensors.

In one example, determining the limitations on the movement of therobotic system include determining relative locations of one or moreobjects in the environment based on the image data and restrictingmovement of the robotic system to avoid colliding with the one or moreobjects.

In one example, determining the limitations includes using simultaneouslocalization and mapping to restrict the movement of the robotic system.

In one example, the method also includes stopping movement of therobotic system responsive to the robotic system moving farther than adesignated, non-zero distance toward the waypoint.

In one example, the final destination of the robotic system is a brakelever of a vehicle.

In one example, the method also includes switching between manualcontrol of the movement of the robotic system and autonomous movement ofthe robotic system based on input received from a control unit disposedoff-board the robotic system.

In one embodiment, a robotic system includes a robotic vehicle having apropulsion system configured to propel the robotic vehicle, one or morestructured light sensors configured to be disposed onboard the roboticvehicle and to obtain point cloud data representative of an externalenvironment, and a controller configured to be disposed onboard therobotic vehicle and to determine a waypoint for the robotic vehicle tomove toward. The waypoint is located between a current location of therobotic vehicle and a brake lever of a rail vehicle. The controller alsois configured to determine limitations on movement of the roboticvehicle toward the waypoint. The limitations are based on the pointcloud data. The controller is configured to control the propulsionsystem to move the robotic vehicle to the waypoint subject to thelimitations on the movement to avoid colliding with one or more objects.

In one example, the controller also is configured to determine one ormore additional waypoints subsequent to the robotic vehicle reaching thewaypoint, determine one or more additional limitations on the movementof the robotic vehicle toward each of the respective additionalwaypoints, and control the propulsion system of the robotic vehicle tosequentially move the robotic vehicle to the one or more additionalwaypoints.

In one example, the controller is configured to determine thelimitations on the movement of the robotic vehicle by determiningrelative locations of one or more objects in the external environmentbased on the point cloud data and restricting movement of the roboticvehicle to avoid colliding with the one or more objects.

In one example, the controller is configured to determine thelimitations using simultaneous localization and mapping to restrict themovement of the robotic vehicle.

In one example, the controller also is configured to stop movement ofthe robotic vehicle responsive to the robotic vehicle moving fartherthan a designated, non-zero distance toward the waypoint.

In one example, the controller also is configured to switch betweenmanual control of the movement of the robotic vehicle and autonomousmovement of the robotic vehicle based on input received from a controlunit disposed off-board the robotic vehicle.

As used herein, an element or step recited in the singular and proceededwith the word “a” or “an” should be understood as not excluding pluralof said elements or steps, unless such exclusion is explicitly stated.Furthermore, references to “one embodiment” of the presently describedsubject matter are not intended to be interpreted as excluding theexistence of additional embodiments that also incorporate the recitedfeatures. Moreover, unless explicitly stated to the contrary,embodiments “comprising” or “having” an element or a plurality ofelements having a particular property may include additional suchelements not having that property.

It is to be understood that the above description is intended to beillustrative, and not restrictive. For example, the above-describedembodiments (and/or aspects thereof) may be used in combination witheach other. In addition, many modifications may be made to adapt aparticular situation or material to the teachings of the subject matterset forth herein without departing from its scope. While the dimensionsand types of materials described herein are intended to define theparameters of the disclosed subject matter, they are by no meanslimiting and are exemplary embodiments. Many other embodiments will beapparent to those of skill in the art upon reviewing the abovedescription. The scope of the subject matter described herein should,therefore, be determined with reference to the appended claims, alongwith the full scope of equivalents to which such claims are entitled. Inthe appended claims, the terms “including” and “in which” are used asthe plain-English equivalents of the respective terms “comprising” and“wherein.” Moreover, in the following claims, the terms “first,”“second,” and “third,” etc. are used merely as labels, and are notintended to impose numerical requirements on their objects. Further, thelimitations of the following claims are not written inmeans-plus-function format and are not intended to be interpreted basedon 35 U.S.C. §112(f), unless and until such claim limitations expresslyuse the phrase “means for” followed by a statement of function void offurther structure.

This written description uses examples to disclose several embodimentsof the subject matter set forth herein, including the best mode, andalso to enable a person of ordinary skill in the art to practice theembodiments of disclosed subject matter, including making and using thedevices or systems and performing the methods. The patentable scope ofthe subject matter described herein is defined by the claims, and mayinclude other examples that occur to those of ordinary skill in the art.Such other examples are intended to be within the scope of the claims ifthey have structural elements that do not differ from the literallanguage of the claims, or if they include equivalent structuralelements with insubstantial differences from the literal languages ofthe claims.

What is claimed is:
 1. A robotic system comprising: a robotic vehiclehaving a propulsion system configured to propel the robotic vehicle; oneor more sensors configured to be disposed onboard the robotic vehicleand to obtain image data representative of an external environment; anda controller configured to be disposed onboard the robotic vehicle andto determine a waypoint for the robotic vehicle to move toward, thewaypoint located between a current location of the robotic vehicle and afinal destination of the robotic vehicle, the controller also configuredto determine limitations on movement of the robotic vehicle toward thewaypoint, wherein the limitations are based on the image data, andwherein the controller is configured to control the propulsion system tomove the robotic vehicle to the waypoint subject to the limitations onthe movement to avoid colliding with one or more objects, wherein thecontroller also is configured to determine one or more additionalwaypoints subsequent to the robotic vehicle reaching the waypoint,determine one or more additional limitations on the movement of therobotic vehicle toward each of the respective additional waypoints, andcontrol the propulsion system of the robotic vehicle to sequentiallymove the robotic vehicle to the one or more additional waypoints.
 2. Therobotic system of claim 1, wherein the one or more sensors areconfigured to obtain a point cloud of the external environment using oneor more structured light sensors.
 3. The robotic system of claim 1,wherein the controller is configured to determine the limitations on themovement of the robotic vehicle by determining relative locations of oneor more objects in the external environment based on the image data andrestricting movement of the robotic vehicle to avoid colliding with theone or more objects.
 4. The robotic system of claim 1, wherein thecontroller is configured to determine the limitations using simultaneouslocalization and mapping to restrict the movement of the roboticvehicle.
 5. The robotic system of claim 1, wherein the controller alsois configured to stop movement of the robotic vehicle responsive to therobotic vehicle moving farther than a designated, non-zero distancetoward the waypoint.
 6. The robotic system of claim 1, wherein the finaldestination of the robotic vehicle is a brake lever of a vehicle.
 7. Therobotic system of claim 1, wherein the controller also is configured toswitch between manual control of the movement of the robotic vehicle andautonomous movement of the robotic vehicle based on input received froma control unit disposed off-board the robotic vehicle.
 8. A methodcomprising: obtaining image data representative of an environmentexternal to a robotic system; determining a waypoint for the roboticsystem to move toward, the waypoint located between a current locationof the robotic system and a final destination of the robotic system;determining limitations on movement of the robotic system toward thewaypoint, wherein the limitations are based on the image data;controlling a propulsion system of the robotic system to move therobotic system to the waypoint subject to the limitations on themovement to avoid colliding with one or more objects; determining one ormore additional waypoints subsequent to the robotic system reaching thewaypoint; determining one or more additional limitations on the movementof the robotic system toward each of the respective additionalwaypoints; and controlling the propulsion system of the robotic systemto sequentially move the robotic system to the one or more additionalwaypoints.
 9. The method of claim 8, wherein obtaining the image dataincludes obtaining a point cloud using one or more structured lightsensors.
 10. The method of claim 8, wherein determining the limitationson the movement of the robotic system include determining relativelocations of one or more objects in the environment based on the imagedata and restricting movement of the robotic system to avoid collidingwith the one or more objects.
 11. The method of claim 8, whereindetermining the limitations includes using simultaneous localization andmapping to restrict the movement of the robotic system.
 12. The methodof claim 8, further comprising stopping movement of the robotic systemresponsive to the robotic system moving farther than a designated,non-zero distance toward the waypoint.
 13. The method of claim 8,wherein the final destination of the robotic system is a brake lever ofa vehicle.
 14. The method of claim 8, further comprising switchingbetween manual control of the movement of the robotic system andautonomous movement of the robotic system based on input received from acontrol unit disposed off-board the robotic system.
 15. A robotic systemcomprising: a robotic vehicle having a propulsion system configured topropel the robotic vehicle; one or more structured light sensorsconfigured to be disposed onboard the robotic vehicle and to obtainpoint cloud data representative of an external environment; and acontroller configured to be disposed onboard the robotic vehicle and todetermine a waypoint for the robotic vehicle to move toward, thewaypoint located between a current location of the robotic vehicle and abrake lever of a vehicle, the controller also configured to determinelimitations on movement of the robotic vehicle toward the waypoint,wherein the limitations are based on the point cloud data, and whereinthe controller is configured to control the propulsion system to movethe robotic vehicle to the waypoint subject to the limitations on themovement to avoid colliding with one or more objects.
 16. The roboticsystem of claim 15, wherein the controller also is configured todetermine one or more additional waypoints subsequent to the roboticvehicle reaching the waypoint, determine one or more additionallimitations on the movement of the robotic vehicle toward each of therespective additional waypoints, and control the propulsion system ofthe robotic vehicle to sequentially move the robotic vehicle to the oneor more additional waypoints.
 17. The robotic system of claim 15,wherein the controller is configured to determine the limitations on themovement of the robotic vehicle by determining relative locations of oneor more objects in the external environment based on the point clouddata and restricting movement of the robotic vehicle to avoid collidingwith the one or more objects.
 18. The robotic system of claim 15,wherein the controller is configured to determine the limitations usingsimultaneous localization and mapping to restrict the movement of therobotic vehicle.
 19. The robotic system of claim 15, wherein thecontroller also is configured to stop movement of the robotic vehicleresponsive to the robotic vehicle moving farther than a designated,non-zero distance toward the waypoint.
 20. The robotic system of claim15, wherein the controller also is configured to switch between manualcontrol of the movement of the robotic vehicle and autonomous movementof the robotic vehicle based on input received from a control unitdisposed off-board the robotic vehicle.